Enabling observability for runtimes

Overview

OpenTelemetry lets you send telemetry data from your runtime and integration process executions to third-party application monitoring tools that support the OpenTelemetry standard. As a runtime owner you can now stream metrics, logs, and traces, in real time, to the monitoring endpoint or OpenTelemetry collector of your choice.

OpenTelemetry is available to all accounts and runtime types, with the exception of cloud attachments, including attachments to Boomi hosted runtime clouds. Cloud attachment-specific observability configuration is planned for a future release.

How it works

Runtime administrators can enable and configure OpenTelemetry for individual runtimes within their account.
The feature is currently configured via the RuntimeObservabilitySettings platform API only. Configuration via the UI is planned for a future release.

The runtime OpenTelemetry configuration provides several basic filters to manage the telemetry data sent by the runtime. Advanced filtering can be performed within the endpoint application or with a collector.

There are three types of observability data.

• Metrics - Periodic snapshot of application statistics. For Boomi, metrics are currently available via JMX metrics.
• Logs - Recording of an event within the application. For Boomi, events are currently captured in log files such as the container log and process log.
• Traces - The execution path of a given request through the application. For Boomi, the closest is the process log or process state.

Learn more about OpenTelemetry and supported vendor tools.

Architecture

The Boomi OpenTelemetry feature works as a plugin within the integration runtime. The lifecycle of the plugin (as with all plugins) is completely managed by the Boomi runtime working together with the Boomi Platform to download, install, configure, and invoke the plugin. It is not an embedded agent or collector.

The telemetry data is communicated or “exported” directly to an endpoint application, which ingests and indexes the data so it can be used within that application. Many endpoint applications consume a mix of telemetry signals, while others are single purpose (e.g. only traces).

You do not need to install a third party agent with the runtime. However you can optionally choose to install a collector. The collector acts as an intermediary, providing a level of buffering/throttling, resiliency, enrichment, and filtering that can be useful in a production environment.

Considerations

• When observability is enabled for a runtime, it will begin transmitting all enabled telemetry data to your endpoint. This could potentially be a large amount of data depending on execution volume. Note that some monitoring tools charge by volume of data collected. You can reduce the volume of data using the available filters within the runtime configuration or filter outside of Boomi in the endpoint application or collector.
• When observability or a specific telemetry type is disabled, no telemetry data is generated.
• Delivery of telemetry data from the runtime to the endpoint application or collector is not guaranteed in the event of extended network interruptions or endpoint application availability. Although use of a third party collector can provide some level of buffering, telemetry data is not queued indefinitely for when connectivity is restored. As such, observability is not intended for critical business transaction tracing. For such tracing, alternative approaches such as emitting key business events as part of integration process design is recommended.

Supported telemetry details

Type	Details
General	All runtime types (basic runtime, runtime cluster, runtime cloud). Cloud attachments are not supported at this time. In clustered runtimes, logs and metrics will be available for each of the cluster nodes. For runtime clouds, runtime owners can collect data from the transient JVMs (forked executions, execution workers, and connection browsers) for cloud attachments.
Protocol	All telemetry data is exported from the runtime via grpc. If your endpoint requires a different protocol such as http or protobuf, use a third party collector to forward data with the desired protocol.
Metrics	For the list of supported metrics, refer to System monitoring with JMX and look for the OpenTelemetry Metric. Note not all JMX metrics are applicable for OpenTelemetry.
Logs	Core runtime logs (container log), per node for clusters. Execution logs (process and document logs). Forked JVMs (forked executions, workers, browsers) for runtime clouds
Traces	All types of process executions (scheduled, manual, listeners) All process modes (general, bridge, and low latency) Nested spans for each step within an execution, with contextual attributes per step type where applicable.